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Abstract  -  This  article  designs  a  first-order  road  map  for 
modeling  research  to  bridge  the  scientific  gap  between 
observations  from  physical  sensor  networks  at  3-50  m  on 
the  one  hand  and  determination  of  covert  tactical 
adversarial  intent  of  individuals  with  deception  and  in 
extensive  clutter  on  the  other.  To  be  successful,  the 
research  needs  to  integrate  kinesiological, 
neurophysiological,  psychological,  and  cognitive  science, 
and  sociocultural  anthropology  and  information  science 
components.  Research  and  development  (R&D)  issues 
that  need  to  be  considered  include  metrics  for  cognitive 
phenomena  and  how  well  detection  systems  work,  data 
sets,  determining  whether  actors  can  provide  sufficient 
verisimilitude  to  create  data  sets,  and  relevant  sensing 
technologies  and  information  fusion  techniques. 
Successful  procedures  may  need  to  include  actively  (but 
unobtrusively)  perturbing  the  situation  in  which  the 
sensing  takes  place  in  order  to  elicit  specific  responses. 
Comprehensive  government  R&D  programs  are  required 
to  promote  rapid  progress. 
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1  Introduction 

The  ability  to  identify  covert  intent  of  individuals  who  may 
be  hostile  would  significantly  improve  asymmetric  counter¬ 
insurgency  and  peace-keeping  operations.  Such  individuals 
are  generally  embedded  in  extensive  “clutter”  of  neutral 
and  friendly  human  beings  and  various  physical  objects.  At 
present,  covert  adversarial  intention  is  identified  through 
judgment  of  soldiers  and  close-range  sensing  and  searching, 
which  often  entail  significant  danger  and  possibly  high 
false-positive  and  false-negative  rates.  Determining  covert 
adversarial  intent  will  help  shift  the  balance  in  operations, 
mission  planning,  training,  and  simulation  from  more  costly 
and  dangerous  sweeping  operations  toward  much  safer 
pinpoint  operations  based  on  refined  estimates  of  people 
from  which  danger  may  come.  Dual-use  civilian  benefits 
will  be  in  crowd  control  and  antidrug,  anticrime,  and 
immigration  enforcement. 

It  has  been  known  since  the  Facial  Action  Coding 
System  (FACS)  created  by  Ekman  and  Friesen  [1]  that  the 
expression  and  micro-expression  of  certain  emotions 
related  to  adversarial  intent  take  place  partially 
involuntarily  through  facial  muscles.  Other  physiological 
actions  such  as  speech,  heart  rate,  respiration  rate,  skin 
temperature,  and  perspiration  can  also  carry  information 
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Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
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about  emotions  related  to  intent,  although  they  are  more 
subject  to  environmental  influence  than  facial  expressions. 
Laser  methods  can  detect  muscle  movement,  heart,  and 
respiration  rate  and  skin  temperature.  Visual  sensors 
provide  information  about  facial  and  body  dynamics. 
Computer  vision  techniques  can  now  automatically  track 
facial  expressions,  eye  movement,  and  gestures.  The  ability 
to  fuse  information,  for  example,  laser  information  with 
visual  information,  and  to  identify  seemingly  hidden 
patterns  is  increasing  rapidly.  Many  of  these  techniques 
require  close-range  sensing/observation,  often  in  a 
controlled  environment  and  at  the  0-2-m  range.  This  is 
suitable  for  airport  screening  but  not  appropriate  for 
asymmetric  defense  scenarios,  where  threats  need  to  be 
detected  at  3  m  and,  preferably,  up  to  50  m.  To  what  extent 
close-range  sensing  techniques  can  be  extended  to  longer 
ranges  is  not  yet  known.  Moreover,  modeling  of  the 
connection  between  emotions  and  intent  is  quite 
incomplete.  Detecting  that  a  person  is  afraid,  for  example, 
does  not  provide  significant  evidence  for  or  against  a 
hypothesis  that  the  person  has  adversarial  intent,  since  both 
friendly  and  adversarial  people  often  have  fear  in 
commonly  encountered  scenarios.  Conversely,  a  person 
with  adversarial  intent  may  show  little  or  no  fear.  While 
there  is  some  understanding  about  how  to  detect  emotions, 
there  is  much  less  understanding  about  how  to  go  further 
into  the  cognitive  realm  and  determine  intent. 

The  fundamental  principles  that  allow  remote  (that  is, 
at  3-50  m)  identification  of  covert  adversarial  intent  based 
on  externally  observable  physical  information  are  not 
known.  Indeed,  the  step  from  recognizing  physical  objects, 
events,  and  patterns  to  recognizing  intent  is  fairly  described 
as  a  scientific  chasm.  Recent  basic  research  related  to 
bridging  this  chasm  includes,  but  is  not  limited  to,  “Future 
Attribute  Screening  Technology  (FAST)”  (Department  of 
Homeland  Security — DHS),  “Violent  Intent  Modeling  and 
Simulation”  (VIMS)  (DHS),  “Detection  of  Intent  through 
Perception  of  Biomotion  Signatures”  and  “Visualization  of 
Belief  Systems”  (U.S.  Army  Research  Laboratory  Human 
Research  and  Engineering  Directorate — ARL/HRED), 
“Remote  and  Passive  ID  of  Electrodermal  Response” 
(Night  Vision  and  Electronic  Sensors  Directorate — 
NVESD),  “Behavioral  Signatures”  and  “Human  MASINT” 
(U.S.  Air  Force  Research  Laboratory — AFRL),  “Hostile 
Intent”  (U.S.  Naval  Research  Laboratory — NRL), 
“Computational  Modeling  of  Adversary  Attitudes  and 
Behaviors”  (Air  Force  Office  of  Scientific  Research — 
AFOSR,  George  Mason  University),  “Dynamic,  Adaptive 
Techniques  for  Adversary  Behavior  Modeling”  (AFOSR, 
University  of  Maryland),  “Human,  Social,  Cultural,  and 
Behavioral  Modeling”  (U.S.  Army  Research  Office — ARO, 
Carnegie  Mellon  University,  University  of  Maryland), 
“Tools  for  Recognizing  Unconscious  Signals  of 
Trustworthiness  Program  (TRUST)”  (Intelligence 
Advanced  Research  Projects  Activity — IARPA),  and 
HERMES,  a  computer  vision  system  for  analyzing  human 
behavior  (Universitat  Autonoma  de  Barcelona, 


http://www.eurekalert.org  /pub_releases  /2010-01/uadb- 
hnc011310.php). 

In  this  article,  we  provide  a  first-order  road  map  for 
modeling  research  to  bridge  the  scientific  gap  between 
observations  from  physical  sensor  networks  at  3-  50  m  on 
the  one  hand  and  determination  of  covert  tactical 
adversarial  intent  of  individuals  with  deception  and  in 
extensive  clutter  on  the  other.  Although  empirical 
observations  and  experiments  will  play  large  supporting 
roles  in  this  research,  the  main  emphasis  is  on  discovery  of 
theoretically  justified  quantitative  predictive  principles 
(models)  and  their  implementation  in  tractable  analytical 
and  computational  procedures.  To  be  successful,  the 
research  needs  to  integrate  kinesiological, 
neurophysiological,  psychological,  and  cognitive  science, 
and  sociocultural  anthropology  and  information  science 
components. 

This  paper  is  based  on  an  ARL  special  report  [2] 
written  shortly  after  the  ARL  Strategic  Directions 
Workshop  “Remote  Detection  of  Covert  Tactical 
Adversarial  Intent  of  Individuals  in  Asymmetric 
Operations”  at  ARL  in  Adelphi,  MD,  on  December  7  and  8, 
2009.  These  documents  describe  directions  of  research  that 
the  participating  agencies  deem  worthy  to  pursue  in  the 
near  future.  There  are  many  related  directions  of  research  of 
interest  to  the  participating  agencies  and  many  other 
agencies,  including,  but  not  limited  to,  close-range  or 
contact  techniques  (e.g.,  for  airport  security  or 
semicooperative  screening),  physical  pattern  recognition, 
behavior  modeling,  predictive  modeling  (“What  will  that 
organization/group/person  do  next?”),  information  from 
non-physics-based  sensors/sources  (human  reports,  media, 
databases,  data  mining,  learning,  World  Wide  Web,  game 
theory,  etc.),  cultural  dependence,  etc.  Research  that  is 
described  here  will  be  influenced  by  and  will  influence 
research  in  these  other  areas,  which  are  important  in  their 
own  right.  However,  to  retain  focus,  this  article  does  not 
treat  these  other  areas. 

Throughout  this  paper,  the  phrase  “remote  detection 
of  adversarial  intent”  will  mean  “remote  (at  3-50  m) 
detection  of  covert  tactical  adversarial  intent  of  individuals 
in  asymmetric  operations.” 

2  The  Current  Situation  and  the  Future 

In  the  early  part  of  the  21st  century,  we  are  in  the  middle  of 
“asymmetric  warfare.”  Asymmetric  warfare,  i.e.,  armed 
conflict  between  a  nation  and  often  faceless,  nameless 
individuals,  and  organizations  that  are  not  affiliated  with 
any  government,  has  occurred  throughout  human  history 
but  has  recently  become  more  prevalent. 

Over  the  past  two  decades,  warfare  has  shifted  from 
traditional  warfare  (i.e.,  wars  of  fire  and  maneuver 
characterized  by  large  force-on-force  actions)  to 
asymmetric  warfare  (i.e.,  wars  of  insurgency)  due  to  many 
factors,  including,  but  not  limited  to,  the  occurrence  of 
foreign  actions  in  the  U.S.  homeland  and  to  operations  in 


Afghanistan  and  Iraq.  There  is  a  need  to  find  individuals 
who  are  hiding  in  civilian  populations  and  remove  them 
without  causing  collateral  damage.  In  order  to  find  these 
individuals,  it  is  necessary  to  determine  their  intent. 

The  concept  “intent”  is  widely  used  in  the  signal 
processing  literature  for  “a  physical  pattern  that  (in  a 
layman’s  view)  allows  a  conclusion  of  intent.”  However, 
the  number  of  false  positives  and  false  negatives  produced 
when  intent  is  interpreted  as  “a  physical  pattern”  is  large. 
Physical  states  such  as  sweating  are  also  poor  indicators  of 
intent.  In  this  paper,  intent  is  a  concept  with  cognitive 
content  that  does  not  need  to  coincide  with  the  seemingly 
obvious  physical  patterns  or  states  often  connected  by  non¬ 
specialists  with  intent.  The  step  from  fused  physics-based 
information  to  a  conclusion  of  cognitive  intent  is  huge.  In 
this  process,  physical  patterns  and  states  may  not 
individually  be  indicative  of  intent.  However,  by 
proceeding  from  the  physical  patterns  and  states  through  a 
fusion  process  that  will  put  “orthogonal”  signals  together 
based  on  cognitive  principles,  intent  can  be  determined. 

Humans  who  are  intending  to  carry  out  preplanned 
violence  have  usually  been  coached  and  prepared  in  a 
manner  not  common  to  daily  life.  Historical  data  on  World 
War  II  kamikaze  pilots  may  be  relevant  to  these 
considerations.  Differentiable  commonalities  may  exist  in 
schooling  and  feelings  of  kinship.  Just  knowing  one’s  task 
and  fate  can  have  consequences  in  biological  motion, 
especially  if  the  violence  is  technologically  enabled  (e.g., 
triggered  bomb  vest).  There  is  evidence  that  observers 
watching  video  clips  on  closed-circuit  television  can 
reliably  detect  individuals  carrying  weapons  by  judging  that 
these  individuals  appear  to  have  higher  levels  of  malaise 
and  restlessness  [5]. 

This  article  focuses  on  remote  detection  of  covert 
tactical  adversarial  intent  of  individuals  in  asymmetric 
operations.  In  future  operations,  this  topic  will,  of  course, 
be  embedded  in  a  larger  framework  that  will  include  use  of 
information  from  databases,  media,  human  sources,  and 
other  non-physics-based  sources,  which  can  operate  at  long 
distances  (greater  than  50  m)  and  long  time  scales  (hours, 
days,  months).  However,  given  the  paucity  of  knowledge  in 
this  area,  it  is  preferable  to  start  from  research  that  involves 
only  physics-based  sensors  and  medium  distances  (3-50 
m). 

A  multitude  of  potentially  fruitful  research  directions 
is  discussed  in  the  next  section.  These  directions  indicate 
that  it  is  fair  to  estimate  that  remote  detection  of  adversarial 
intent  is  indeed  feasible.  Success,  however,  will  depend  on 
the  details  of  how  the  already  existing  building  blocks  plus 
new  building  blocks  can  be  put  together. 

3  Research  and  Development  Directions 

Our  objective  here  is  to  describe  R&D  directions  that  will 
eventually  bridge  the  gap  between  physical  phenomena  and 
cognitive  intent  using  scientifically-based  (rather  than 
layman’ s-experience-based)  principles.  Given  the  paucity 
of  knowledge  in  the  area,  we  cannot  always  demand  well- 


established  principles.  Hypothesized  principles  may  have  to 
be  used  when  established  principles  are  unavailable. 

3.1  Cognitive/Perceptual  Phenomena 

In  research  on  sensing  adversarial  intent,  a  basic 
assumption  is  that  every  brain  action  or  state  has  a 
corresponding  effect  or  state,  however  diminutive  or 
hidden,  in  the  body  and  that  the  effect  or  state  in  the  body  is 
amenable  to  some  sort  of  sensing,  even  if  not  remote.  Can 
quantitative  connections  between  cognitive  states  and 
detectable  signals  be  identified?  Is  passive  observation 
sufficient  or  is  it  useful/necessary  to  actively  (but 
unobtrusively)  introduce  signals/perturbations  in  the 
observation  process  to  gather  sufficient  information? 
“Active  elicitation”  of  information  may  be  a  fruitful  area 
for  research. 

One  should  note  that  superficial  “profiling”  seems  to 
fail.  The  Transportation  Security  Agency  (TSA)  has  noted 
that  random  screening  produces  more  valid  “hits”  than 
superficial  profiling.  However,  this  experience  may  not  be 
applicable  without  caveats  to  many  defense  situations.  Can 
cognitive  state  be  quantified  and  quantitatively  related  to 
detectable  signals,  or  is  this  even  needed?  Is  physiological 
state  enough  to  determine  that  a  person  is  “anomalous”  and 
has  an  adversarial  “plan?” 

What  are  the  effects  of  people  who  are  actually 
innocent  bystanders  but  have  knowledge  that  can 
potentially  jeopardize  them  and  their  family?  Can  research 
be  done  on  bystanders?  What  are  the  cues  that  a  bystander 
gives  off  if  he  or  she  knows  something  about  a  plan?  Are 
these  cues  detected  by  “expert  human  detectors?”  Can  these 
cues  be  automatically  detected  by  sensors? 

Exploration  of  enemy  tactics,  techniques,  and 
procedures  as  contextual  factors  for  determining  intent 
should  be  considered.  Most  individuals  intending  to  do 
harm  have  gone  through  at  least  some  type  of  training  and 
practice.  There  is  often  a  need  to  actively  induce  or  ramp- 
up  a  response  from  individuals  so  that  it  can  be  sensed.  By 
actively  inducing  a  response,  the  flow  of  a  plan  will  be 
altered  and  perhaps  more  easily  observable.  Those  who 
have  a  plan  may  be  affected  more  by  even  normal  delays 
than  others.  People  with  a  plan  may  get  increasingly  angry 
with  delays,  whereas  others  may  just  have  ordinary 
annoyance  but  not  anger.  The  TSA  SPOT  program  is 
investigating  such  issues. 

After  the  beginning  of  a  stimulus,  physiological  cues 
in  some/many  processes  can  peak  in  5-10  s.  The  human 
body  tends  to  have  many  non-specific  responses  (i.e., 
physiological  responses  that  are  part  of  the  homeostatic  or 
other  internal  state  of  the  body  unrelated  to  external 
cognitive  stimuli).  In  order  to  understand  which 
physiological  cues  relate  to  which  stimulus,  it  is  necessary 
to  know  the  timing  of  the  stimulus.  Can  a  person’s 
cognitive  load  be  probed  in  time?  If  so,  these  probes  need 
to  be  culturally  appropriate  and  relatively  inconspicuous. 
How  and  when  can  previously  rehearsed  patterns  be 
perturbed?  It  will  be  important  to  develop  good  stimuli  for 


evoking  a  response  that  might  be  an  idealized  adversarial 
intent  cognitive  state,  emotion,  or  arousal  level.  There  may 
be  differences  due  to  different  situations.  Those  carrying 
bomb  vests  implying  self-immolation  can  have  a  different 
cognitive  set  (with  associated  biomarkers)  than  those 
controlling  remotely  triggered  explosive  devices.  Research 
about  how  one  actively  (and  unobtrusively)  controls  the 
situation  so  that  threat  indicators  are  likely  to  be  expressed 
and  thereby  achieves  subject  engagement  and  behavioral 
authenticity  is  needed.  Given  the  enormity  of  the  topic  and 
its  currently  amorphous  state,  perhaps  an  iterative  approach 
to  experimentation,  in  which  scenarios  are  initially  kept 
relatively  simplistic  and  then  experimental  complexity  is 
increased  as  knowledge  is  gained,  is  best. 

Research  on  states  of  mind  related  to  the  timing  of  the 
event  can  be  beneficial.  For  example,  if  an  act  and  the 
associated  verbiage  to  be  used  if  interrogated  have  been 
well  rehearsed,  then  there  might  still  be  some  variability  in 
responses  up  until  very  close  to  the  act.  At  that  point,  there 
could  be  a  reduction  in  normal  physiological 
responsiveness  but  more  of  a  response  to  an  unexpected 
event  delaying  the  plan.  Research  related  to  planning  and 
disruption  to  plans  might  be  helpful  to  distinguish  the  effect 
of  the  type  of  plan  on  the  response  to  disruption  or  delay. 
Much  of  what  may  have  to  be  done,  without  the  benefit  of 
an  interview/interrogation,  is  to  induce  a  response.  This 
response  can  be  cognitive  or  behavioral  and  will  be  biased 
by  cultural  constructs.  Stimulus-response  operational 
design  in  a  complete  theoretical  framework  is  likely  to  be 
advantageous.  However,  the  likely  difficulty  in  constructing 
a  suitable  theoretical  framework  suggests  that  beginning 
with  an  empirical  and  observational  approach  may  be 
advisable. 

Differentiation  needs  to  be  made  between 
psychological  issues  (ideologues,  disaffected,  etc.)  and 
psychopathological  issues  (not  sociopathic).  Clinical  data 
may  not  be  applicable  as  physiological  indicators.  For 
example,  sociopaths  may  be  emotionally  numb.  Further, 
there  could  be  possible  links  to  religious  fanaticism  and  the 
associated  training  involved  (cf.,  for  example,  the  El 
Alamut  Assassins  http://www.alamut.com/subi/ideologies 
/alamut/secDoctrines.html).  Can  we  identify  the  different 
types  of  terrorist  psychologies?  Are  there  significant 
differences  between  the  person  who  totally  believes  he  or 
she  is  carrying  out  an  important  duty  vs.  the  person  who  has 
been  forced  into  compliance  with  the  plan  through  threats 
and  intimidation? 

Another  important  and  often  overlooked  concept  is 
measuring  the  problem.  Metrics  for  cognitive  phenomena 
and  for  how  well  detection  systems  work  are  needed.  The 
metrics  themselves  may  be  computationally  intensive  but 
should  run  seamlessly  in  the  background,  invisible  to  the 
user.  Results  should  be  easily  interpretable  and  not  lend 
themselves  to  a  “so  what”  response  by  the  user.  Often,  new 
metrics  are  developed  in  conjunction  with  user 
communities,  which  usually  ensures  that  such  metrics  are 
practically  useful.  In  addition  to  being  practically  useful, 


however,  the  metrics  need  to  be  computationally  feasible 
(not  combinatorially  expensive)  and  mathematically 
justified.  Whatever  metrics  are  proposed  should  be  justified 
not  based  on  traditional  use  of  the  metrics  in  other  areas, 
successful  as  that  use  may  be,  but  rather  on  the  basis  of 
human  goals  in  the  remote  detection  of  covert  tactical 
adversarial  intent.  In  the  past,  metrics  based  on  information 
theory  have  been  recommended  for  use  in  the  fusion 
process.  Information-theoretic  metrics  were  designed  for 
non-biological  physical  processes.  Evidence  that  they  are 
applicable  to  the  physiological,  psychological,  and 
cognitive  processes  of  interest  here  could  perhaps  be 
adduced  but  is  unavailable. 

Another  major  issue  is  the  development  of  data  sets. 
Can  “method  acting”  (or  any  other  school  of  acting) 
provide  sufficient  verisimilitude  on  all  scales,  including 
emotive/biochemical  (sweat,  breath,  body  habitus, 
kinesiology)  to  permit  its  use  as  a  surrogate  for  “real”  data? 
If  so,  the  creation  of  data  sets,  while  still  expensive,  will  be 
less  expensive.  If  enacted  experiments  cannot  provide  data 
that  matches  data  of  “real”  situations,  the  expense  and 
uncertainty  will  be  larger.  It  is  often  expensive  and  difficult 
to  obtain  large  amounts  of  data  on  “real”  situations.  The 
ground  truth,  i.e.,  the  true  intent  of  the  subject,  may  be 
difficult  to  determine  because  of  deception  or 
disappearance  of  the  subject.  The  virtual  reality  community 
has  a  well-established  database  for  determining  participant 
engagement;  this  database  should  be  explored  for  its 
applicability  here.  Again,  a  primary  concern  is  authenticity 
of  the  data  gathered  when  the  input  is  produced  by  acting. 

An  alternative  is  to  examine  the  principles  of 
immersive  game  system  design  and  human  games. 
Automatic  detection  of  adversarial  intent  can  benefit  from 
using  methods  that  human  experts  use,  including,  but  not 
limited  to,  criminological  strategies  of  presenting  photos  of 
a  crime  scene,  presenting  mug  shot  albums,  exposing 
subjects  to  news  clips  or  headline  news,  filling  out  surveys 
or  forms,  and  delaying  forward  movement.  There  is, 
however,  an  open  question  about  whether  such  expertise  is 
transferable  without  extensive  personal  immersion  within  a 
conflict  environment.  Also,  whether  these  can  be  adapted 
for  the  desired  scenario  (3-50-m  distance)  will  need  to  be 
examined. 

The  difficulty  in  obtaining  ground  truth  for  adversarial 
intent  detection  is  an  issue.  The  occurrence  of  adversarial 
intent,  less  than  1/1000,  is  also  an  issue.  This  makes  a  non- 
normative  approach  to  statistics  important.  Each  individual 
may  have  unique  responses  relevant  to  his  or  her 
adversarial  intent.  Are  there  special  populations  that  can  be 
ethically  tapped  to  provide  appropriate  cognitive  or 
psychological  constructs?  Can  incentives  provide  the 
amount  of  “buy-in”  needed  for  a  valid  proxy  of  a  “bad 
guy?”  Since  we  don’t  know  what  a  “real  bad  guy”  with 
adversarial  intent  may  be  like,  perhaps  we  need  to  know  in 
exquisite  depth  what  a  good  guy  looks  like. 


3.2  Sensing 

Individuals  with  adversarial  intent  will  experience 
physiological  changes  and  possibly  display  altered  motions 
and  behaviors  that  may  be  detected.  In  this  section,  we 
discuss  physics-based  sensors  that  may  be  able  to  remotely 
detect  indicators  for  subsequent  fusion  to  determine  intent. 
Physics-based  sensors  will  provide  the  input  for  a  cognitive 
approach  to  sensing  adversarial  intent  and  offer  the 
advantage  of  a  quantitative  assessment. 

The  goal  is  to  identify  measureable  physiological, 
biochemical,  genetic,  or  other  types  of  indicators  that  are 
correlated  with  an  underlying  elevated  level  of  stress;  a 
determination  to  carry  out  a  plan  with  adversarial 
consequences  and  other  less  temporal  predictors  of  violent 
activity  including,  but  not  limited  to,  social 
disconnectedness;  and  an  embracing  or  acceptance  of 
violence  and  death.  Psychological,  physiological,  genetic, 
and  biochemical  factors  that  can  correlate  with  social 
disconnectivity  and  a  willingness  to  commit  violence 
include  aggression,  impulsivity,  autism,  post-traumatic 
stress  disorder  (PTSD),  unattractiveness,  variants  of 
chromosomal  copy  number,  and  mutations  in  specific 
genes. 

Informative  biological  parameters  that  can  be 
measured  include  the  following: 

•  Posture 

•  Posture  rigidity 

•  Heartbeat  waveform 

•  Heart  rate 

•  Breath  rate,  volume  approximation,  patterns,  anomalies 

•  Wheezing,  coughing,  gasping 

•  Blood  pressure  trends:  waveform  shape  and  transit  time 

•  Pulse-wave  velocity  giving  a  beat-by-beat 
approximation  of  blood  pressure 

•  Movement:  fidgeting,  remaining  still,  shaking, 
shivering,  having  spasms 

•  Body  stiffness,  muscle  tension,  resonant  frequency  of 
body  movement 

•  Voice  stress  analysis  and  voice  onset  timing 

•  Gastrointestinal  distress,  bowel  sounds 

•  Reluctance  to  engage  socially:  distance  from  others, 
response  to  attempts  to  engage  verbally 

•  Observation  tendencies  of  subject,  eye-glancing,  head 
turning,  situational  awareness 

•  People  whose  actions  are  coordinated  or  who  are 
actively  avoiding  each  other 

•  Exposure  to  bomb-making  materials/chemicals 

•  Hyperthermia  from  stress  (generally  expressed  in  the 
face,  palms  of  the  hands,  and  soles  of  the  feet) 

•  Gait  as  indicators  of  stiffness  (stress)  or  carrying  a  load 
or  wearing  protective  clothing 

•  Breath  biochemistry 

•  Microbiological  organisms  on  skin  or  clothing 
Attention  should  also  be  given  to  screening  people  in 
vehicles.  Some  potential  informative  and  measureable 
signals  include  the  following: 


•  Body  and  movement  rigidity/stiffness  as  a  measure  of 
stress,  probably  more  informative  if  measured  in  response 
to  unexpected  perturbation  such  as  a  unusual  speed  bump, 
unexpected  timing  of  red  light,  provocative  noise,  or  other 
stressor 

•  Acoustic  signals:  If  one  can  hear  what  is  being  said  or 
happening  in  the  vehicle,  are  the  occupants  making  small 
talk,  silent,  listening  to  the  radio,  or  praying? 

•  Lip  and  facial  movements:  Are  the  occupants  talking, 
praying,  silent,  listening  to  the  radio,  or  listening  for  outside 
cues? 

Body-contacting  sensors  that  assess  human 
physiology,  such  as  those  used  for  polygraphy,  biometric 
identification  and  medical  diagnosis,  are  common. 
However,  using  these  sensors  at  stand-off  distances 
presents  both  physics-based  and  operational  hurdles.  Many 
of  the  current  sensors  cannot  operate  at  even  modest 
distances  (1  m)  away  from  human  bodies.  Ongoing  R&D  is 
dramatically  increasing  sensor  sensitivity,  bandwidth,  field 
of  regard,  and  many  other  characteristics  but  still  may  not 
be  able  to  overcome  many  of  the  technical  barriers 
associated  with  remote  intent  assessment.  Many  of  the 
sensors  mentioned  in  this  section  may  not  be  ready  for 
immediate  application  but  may  offer  significant  benefits  if 
sensor  system  maturity  advances  quickly. 

We  attempt  to  highlight  sensing  technologies  that  we 
believe  can  measure  useful  data  for  assessing  adversarial 
intent.  Passive  and  active  sensors  can  be  used  to  remotely 
characterize  a  human’s  physical  features,  clothing  and 
equipment,  and  their  interaction  with  the  immediate 
surrounding  environment.  Passive  sensors  do  not  emit  a 
signal  and  rely  on  a  target’s  emission  in  some  sensing 
domain.  Active  sensors  emit  a  signal  that  interacts  with  the 
target  in  a  known  manner  to  produce  a  return  that  is 
quantifiable  and  related  to  the  stimulus.  Sensor  fusion  can 
combine  multiple  inputs  and  domains  to  enhance  features 
and  remove  noise  or  interferes.  Redundancy  of  diverse 
sensor  modalities  will  help  corroborate  physiological 
parameters  in  noise  as  well  as  alternative  inputs  for  fusion 
algorithms.  Some  potential  sensor  domains  and  expected 
contributions  are  as  follows: 

•  Imagers,  in  general:  Imagers  with  related  image- 
processing  hardware  and  software  can  assess  shape  and 
contrast  and  change  in  single  images  and  multiple  frames. 
Change  detection  algorithms  detect,  track,  and/or  assess 
movements  and  establish  “normal”  traits,  gait,  movement 
tracking,  surveillance  activity,  facial  expression,  emotion, 
phenotypic  patterns,  and  body  language.  Imagers  provide 
data  for  extracting  behaviors  and  interactions  among  people 
in  the  field  of  view  through  pattern  recognition.  All  of  these 
capabilities  can  be  implemented  using  thermal, 
hyperspectral,  visible,  and  narrowband  imagery. 

•  Thermal  imagers:  Passive  thermal  imagery  can  monitor 
overall  surface  temperature  patterns,  radiometric  levels, 
facial  thermal  patterns,  capillary  dilation  effects, 
bombs/weapons,  clothing  thickness,  and  specific  body 
features. 


•  Hyperspectral  imagers:  Passive  multiband  imagers  are 
optimized  to  select  specific  wavelengths  of  interest  related 
to  particular  optical  phenomena  or  material.  Narrowband 
imagery  can  focus  on  sweat,  subsurface  blood-flow,  and 
particular  types  of  clothing  or  equipment. 

•  Visible  bandwidth  imagers:  These  imagers  establish 
“normal”  visible  appearance,  identifiable  traits  for 
biometric  ID,  facial  expressions,  skin-tone  changes, 
phenotypic  patterns,  and  body  language. 

•  Laser  Doppler  vibrometry:  Active  laser  interrogation  of 
skin  or  body  surfaces  reveals  vibrational  cues  related  to 
heartbeats,  breaths,  body  resonances,  muscle  stiffness, 
voice,  and  voice  stress. 

•  E-Field:  Passive  free-space  electrodes  (capacitively 
coupled)  can  detect  electrical  activity  of  the  heart,  brain, 
muscles,  and  hidden  electronic  devices. 

•  Radar:  Active  radar  can  detect  and  track  human  gait, 
heartbeats,  breaths,  radar  cross  section  (RCS),  and  arm/leg 
movements. 

•  Ladar:  Active  laser  radar  produces  3D  imagery  related 
to  “hostile”  stance,  3D  posture,  gait  dynamics,  facial 
recognition,  carrying  of  backpacks,  and  unnatural  frame 
proportions. 

•  Gas  chromatography:  Active  collection  of  exhaled  gas, 
such  as  through  a  suction  tube  near  a  microphone  or  portal, 
can  provide  a  sample  for  trace  gas  analysis,  chemical 
emission,  odor,  genetic  material,  and  biochemistry 
assessment. 

•  Genetics  of  prokaryotic  microorganisms:  Genetic 
mutation  of  prokaryotic  microorganisms  on  or  in  humans  is 
environmentally  influenced.  DNA  mutates  in  prokaryotes  at 
a  rate  of  approximately  1  in  300  nucleotides  per  generation; 
generations  can  be  as  short  as  10  min.  If  the  mutation 
patterns  were  untangled,  the  resulting  information  could 
reveal  where  the  organism  has  been  and  what  it  has  been 
exposed  to.  Mutation  and  mutation  rates  of  prokaryotic 
organisms  on  or  in  humans  might  thus  help  determine 
where  humans  have  recently  been,  what  they  have  been 
eating,  whether  they  have  had  unusual  exposures  to  man¬ 
made  or  other  toxins  or  mutagens  and,  in  general,  whether 
the  patterns  in  their  microorganisms’  DNA  is  consistent 
with  their  purported  history,  identity,  and  activities.  In 
addition,  the  relative  fitness  of  different  species  of 
prokaryotes  depends  on  the  environment;  the  environment 
of  prokaryotes  growing  in  or  on  humans  is  influenced  by 
that  human’s  activity.  The  relative  abundance  of  different 
species  of  prokaryotes  on  humans  is  also  reflective  of  that 
human’s  environment  and  could  be  exploited  to  reveal 
information  about  that  person’s  activities. 

•  Chemical  Sensors:  Laser-induced  fluorescence,  i.e., 
active  laser  illumination  with  passive  response  monitoring 
(either  fluorescent  imaging  or  non-imaging  amplitude  of  a 
particular  wavelength),  can  be  used  to  inspect  a  portion  of 
the  body  for  sweat  or  other  compounds  (salivary  amylase  or 
cortisol  in  the  mouth  or  in  the  breath).  Laser-induced 
fluorescence,  laser-induced  breakdown  spectroscopy 


(LIBS),  and  Raman  spectroscopy  can  be  used  to  inspect 
body  and  clothing  for  traces  of  explosive  chemicals. 

•  Photoacoustics:  Active  laser  stimulation  of  a  test  area 
and  acoustic  analysis  of  induced  resonance  can  give 
indications  of  trace  gas  from  the  mouth,  biological  markers, 
and  chemical  residue  on  skin. 

•  Retroreflection:  Active  laser  illumination  and  passive 
imaging  of  the  collimated  returns  can  be  used  for  detection 
of  optics/video  cameras  used  by  a  target  and  can  also  detect 
naked  eyes  in  close  proximity  for  staring  or  eye  tracking. 

•  Seismic:  Passive  sensing  of  ground  or  floor  vibrations 
can  be  used  to  assess  gait  anomalies  from  concealed  objects 
and  weight  anomalies  and  for  motion  tracking.  Active 
stimulation  can  induce  movement  of  hidden  objects. 

•  Magnetics:  Passive  magnetic  sensing  can  detect  hidden 
objects  that  might  indicate  intent,  such  as  a  pistol,  knife,  or 
explosive  initiator  component. 

On  a  slightly  higher  level,  one  could  measure  the  following: 

•  Physical  evidence  of  psychological  traits:  Physical 
evidence  of  psychological  traits  of  interest  (PTSD,  autism, 
antisocial  behavior,  embracing  violence,  indifference  to 
human  death,  etc.)  could  be  measured.  One  would  have  to 
develop  an  enabling  database  for  phenotypic-to- 
psychological  correlation. 

Operational  considerations  include  covertness  of 
stand-off  sensing,  amount  of  clothing  that  might  mask 
measureable  parameters,  interfering  signals  from  motion 
artifacts,  environmental  considerations,  ambient  noise, 
speech,  and  the  dynamic  movement  of  the  subject  or  sensor 
system.  If  a  particular  assessment  on  an  individual  requires 
a  comparison  with  some  preexisting  “baseline,”  a  plan 
needs  to  be  established  to  access  the  baseline  in  the 
operational  environments  being  discussed.  An  obvious 
objective  is  to  process  the  data  in  real  time  to  show  states 
and  changes  in  measured  parameters.  However,  there  may 
be  a  requirement  to  log  raw  data  for  post-processing  for 
complex  analysis  of  high-bandwidth  imagery  or 
multichannel  fusion. 

3.3  Information  Fusion 

Remote  human  detection  with  high  classification  and 
determination  of  intent  over  ranges  up  to  50  m  is  a 
challenging  task.  Humans  engaged  in  planning  a  threat 
carefully  plan  their  mission,  rehearse,  and  take 
precautionary  measures  to  hide  and  disguise  their  intention 
as  a  normal  event.  Passive  measurement  of  observable 
physical  items  (gait,  arm  swings,  and  posture — carrying  a 
heavy  load  might  alter  the  way  a  human  walks)  or  a  vehicle 
(determining  the  weight  it  is  carrying — abnormal  RPMs) 
can  be  accomplished.  In  addition,  one  can  introduce  a 
stimulus  that  generates  an  element  of  surprise  that  alters  the 
rehearsed  plan  of  action.  A  human  might,  in  response, 
conduct  unrehearsed  sudden  irregular  motion.  Such 
behaviors  may  help  differentiate  threats  from  non-threats 
when  fused  with  other  “orthogonal”  information  from  other 
sensing  modalities. 


The  Joint  Directors  of  Laboratories  (JDL)  Data  Fusion 
Model  is  the  most  widely  used  method  for  categorizing 
information  fusion-related  functions  [4].  The  information 
fusion  process  involves  combining  information — in  the 
broadest  sense — to  estimate  or  predict  the  state  of  some 
aspect  of  the  universe.  Although  there  are  many  criticisms 
of  the  JDL  model  and  many  competing  models,  the  JDL 
model  has,  in  general,  withstood  the  test  of  time.  Most  of 
the  fusion  community  has  accepted  the  JDL  fusion  levels. 
There  are  frameworks  that  extend  and  more  practically 
functionalize  the  JDL  fusion  model.  An  example  is  the 
Contextual  Fusion  Model  [3,4],  which  includes  context  and 
handles  input  derived  from  both  hard  and  soft  sensors  in 
addition  to  producing  a  more  specific  functionalization  of 
the  desired  fusion  process  for  implementation. 

Here,  we  include  consideration  of  informational  and 
cognitive/perceptual  states  and  physical  states  that  were  not 
the  main  focus  of  the  original  JDL  model.  The  extent  to 
which  the  JDL  model  can  be  extended  to  informational  and 
cognitive/perceptual  states  is  not  yet  known.  The  answer  to 
this  question  hinges,  in  part,  on  issues  such  as 
representation  of  uncertainty  that  were  not  previously 
considered.  In  the  next  paragraph,  we  describe  the  issues 
related  to  representation  of  uncertainty  as  they  are  now 
appearing  in  the  context  of  information  fusion. 

There  is  a  question  about  whether  and  to  what  extent 
classical  probability  theory  (typically,  Bayesian  theory)  is 
applicable  for  fusion  processes  involving  cognitive 
phenomena.  There  is  wide  agreement  that  information  from 
physical  sensors  can  be  assessed  and  fused  using  classical 
probability  measures.  When  cognitive  phenomena  are 
considered,  however,  other  representations  of  uncertainty, 
ones  less  dependent  on  statistical  assumptions  and  more 
able  to  handle  data  of  varying  types  and  sparse  data,  may  be 
needed.  Consider  the  information  processing  at  the 
cognitive  level  as  a  system  of  subsystems  (i.e.,  local 
processing  in  physical  devices,  information  fusion  in  the 
network,  etc.).  Uncertainty  is  propagated  upwards. 
Uncertainty  may  be  in  the  form  of  the  pedigree  of 
information,  degraded  sensing,  digital  communications 
degradation/collisions,  etc.  As  these  uncertainties  are 
propagated  upwards,  the  performance  of  the  higher-level 
system  may  be  adversely  affected.  It  is  possible  that  the 
higher-level  uncertainties  will  not  be  described 
appropriately  by  probabilities  (which  might  have  been 
inappropriately  “summed”  in  prior  efforts).  There  are  many 
alternatives  to  classical  probabilistic  (often  Bayesian) 
methods.  Dezert-Smarandache  theory  has  had  success  in 
robotics,  and  the  Dempster-Shafer  theory  has  been  applied 
in  many  areas  and  has  been  useful  for  “real-life”  problems. 
The  Transferable-Belief  Model  (TBM)  has  been 
successfully  used  in  marketing,  and  Analysis  of  Competing 
Hypotheses  (ACH)  has  been  used  in  human  intelligence. 
All  of  these  methods  have  had  success  in  additional  areas. 
Possibility  theory  is  also  a  candidate  for  assessing 
propagated  uncertainties.  Bosse  et  al.  [7]  discuss  various 
methods  for  representing  uncertainty  in  fusion  processes. 


The  emphasis  here  has  been  on  the  fusion  of 
information  from  physics-based  sensors  because  such 
fusion  is  currently  not  yet  feasible  and  designing  it  is  a 
huge  task  in  itself.  In  future  operations,  all-source  fusion 
(“hard/soft  fusion”),  where  the  input  comes  from  all 
available  sources  (physical  sensors,  informational 
resources,  human  reports,  etc.),  will  certainly  be  required. 
In  all  types  of  fusion,  the  choice  of  the  representation  of 
uncertainty  needs  to  be  coordinated  with  the  fundamental 
science  of  the  phenomena. 

4  Coordination 

The  infrastructure  for  R&D  for  remote  detection  of 
adversarial  intent  is  already  in  place  in  academia, 
government  laboratories,  and  industry.  The  way  in  which 
researchers  and  developers  in  various  disciplines  and 
organizations  collaborate  to  pursue  larger  goals  will  be  a 
large  factor  in  determining  the  rapidity  with  which  R&D  is 
accomplished.  Research  in  remote  detection  of  adversarial 
intent  involves  cognitive  science,  psychology,  physiology, 
and  behavioral  science.  Relevant  areas  include  clinical 
psychology,  cognitive  neuroscience,  communications, 
computer  and  information  science,  criminal  science, 
decision  science,  developmental  psychology,  ethics, 
geography,  human  cognition,  industrial  organization 
psychology,  linguistics  and  computational  linguistics, 
perception  psychology,  physical  and  cultural  anthropology, 
psychophysiological  psychology,  risk  and  risk 
management,  social  psychology,  and  sociology. 

4,1  Balance  between  Near-Term 

Development  and  Long-Term  Research 

In  any  long-term  research  project,  there  is  a  multitude  of 
near-term  issues  that  impact  the  conduct  of  research.  On 
occasion,  these  near-term  issues  could  hinder  progress 
toward  long-term  solutions.  At  other  times,  the  near-term 
issues  aid  progress  toward  long-term  solutions  by  drawing 
attention  to  unforeseen  problems  that,  if  recognized  and 
investigated  early  on,  significantly  shorten  the  path  to  the 
long-term  goal.  The  options  of  focusing  exclusively  on 
long-term  research  without  recognizing  the  importance  of 
near-term  issues  and  focusing  only  on  near-term  issues 
without  cognizance  of  the  need  for  long-term  research  are 
both  less  than  optimal.  Government,  societal,  and  scientific 
needs  require  that  near-term  benefits  and  long-term  goals 
be  balanced  against  each  other. 

The  traditional  technology  transfer  paradigm  of 
scientists,  engineers,  and  mathematicians  making  basic 
discoveries  in  their  research  and  passing  them  on  to 
development  personnel  for  implementation  is  no  longer  the 
only  or  best  option  for  R&D.  Increasingly,  researchers  are 
being  called  upon  to  interact  and  collaborate  with 
development  personnel  to  reduce  the  time  necessary  to 
build  operational  systems.  It  is  in  a  framework  of  two-way 
technical  collaboration  between  basic  researchers  and 


development  personnel  that  issues  impacting  long-term 
development  of  a  system  can  be  identified  earliest  and 
solved  most  efficiently.  A  successful  basic  research 
program  on  remote  detection  of  covert  tactical  adversarial 
intent  of  individuals  in  asymmetric  operations  will  be  a  set 
of  interdependent  projects  linked  interactively  with 
development  programs.  The  urgency  of  the  need  suggests 
strongly  that  applied  R&D  needs  to  begin  before  basic 
research  is  complete.  In  addition,  applied  R&D  will  likely 
generate  “real  data”  on  its  way  to  developing  specific 
systems  and  will  thereby  benefit  basic  research,  which 
currently  has  very  little  access  to  “real  data”  and 
consequently  is  hampered  in  the  development  of  relevant 
theory.  Traditionally,  basic  research  has  been  achieved 
either  by  chance  or  because  of  a  demonstrated  need  over  a 
long  period  of  time.  There  is  currently  a  great  need  to  carry 
out  basic  research  and  applied  R&D  hand-in-hand  so  that 
the  entire  problem  is  solved  and  the  time  taken  to  produce  a 
deployable  solution  is  shortened. 

4.2  Collaboration  among  Government, 
Academia,  and  Industry 

Currently,  there  are  many  efforts  in  and  bordering  on  the 
areas  discussed  in  Section  3.  These  efforts  are  supported 
by  many  different  agencies  and  coordinated  through  formal 
and  informal  meetings  of  investigators  at  working  groups, 
conferences,  and  other  events.  Some  of  these  efforts  by 
DHS,  ARL/HRED,  ARL/ARO,  NVESD,  AFRL,  NRL, 
AFOSR,  and  IARPA  are  mentioned  in  the  Introduction. 
These  and  future  efforts  in  the  area  described  in  this  paper 
are  being  coordinated  among  interested  agencies  with  the 
objective  of  creating  connected,  complementary  projects 
that  accelerate  the  research  overall.  Consistent  with  the 
approach  mentioned  in  Subsection  4.1,  these  efforts  contain 
interconnected  basic  research,  applied  research,  and 
development  components. 

5  A  Path  to  the  Future 

In  the  article,  we  have  summarized  the  state  of  the  art  in 
remote  detection  of  adversarial  intent  and  have  pointed  out 
the  need  for  coordinating  R&D  in  many  dimensions, 
including  1)  cognitive/perceptual  phenomena,  sensing,  and 
information  fusion,  2)  near-term  development  and  long¬ 
term  research,  and  3)  different  types  of  organizations 
(government,  academia,  industry).  The  U.S.  Government 
expects  to  support  R&D  programs  with  the  objective  of 
producing  theoretically  founded  designs  of  a  prototype 
system  for  remote  detection  of  covert  tactical  adversarial 
intent  of  individuals  in  asymmetric  operations.  The 
Government  also  expects  to  provide  broad  support  for 
academic  and  industrial  efforts  in  remote  detection  of 
adversarial  intent  and  in  areas  (such  as  linkage  of  these 
systems  with  databases,  media,  and  human  input)  that  are 
useful  for  larger  systems  of  systems.  Other  governments 
have  expressed  similar  plans.  Research  on  winning  the 


hearts  and  minds  of  populations  could  pay  off  well  in 
reducing  the  occurrence  of  adversarial  intent  [8,9].  The 
reduction  will,  however,  never  be  zero,  and  the  research 
described  in  this  paper  will  always  be  needed. 

Dual  uses  of  remote  detection  of  adversarial  intent  in 
the  civilian  economy  include  crowd  control,  antidrug  and 
anticrime  operations,  border  security,  and  ensuring  the 
security  of  government  and  private  personnel  and  property. 
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